Electrical Engineering and Computer Science Department Technical Report NWU - EECS - 10 - 08 September 13 , 2010 Symbiotic

Virtualization has now become ubiquitous, especially in large-scale data centers. Significant inroads have also be made into high performance computing and adaptive systems, areas where I have focused in the Virtuoso and V3VEE projects. The rapid adoption of virtualization in all of these areas is in no small part due to the ability of full system virtualization to adapt existing OSes to virtual environments with no changes to the OS implementation. On the other hand, paravirtualization, which does require deep OS changes, has been demonstrated to have significant performance and functionality benefits. Beyond paravirtualization, researchers are now discussing other ways to rethink OS design for the virtualization age. One fundamental problem with existing virtualization architectures is that the interfaces they provide to a guest environment exist at a very low level and do not expose high level semantic information. This has created a situation where underlying VMMs often have very little knowledge about the architecture, behavior, or needs of a guest VM. This situation has come to be described as the semantic gap. Furthermore, existing architectures are designed such that obtaining this information is extremely difficult. In order for virtualization to reach its true potential, this problem must be addressed. This effort was partially supported by the National Science Foundation (NSF) via grants CNS-0709168 and CNS-0707365, and by the Department of Energy (DOE) via Sandia National Laboratories (SNL) and Oak Ridge National Laboratory (ORNL), as well as a Symantec Research Labs Fellowship. My dissertation focuses on symbiotic virtualization, a new approach to designing virtualized systems that are capable of fully bridging the semantic gap. Symbiotic virtualization bridges the semantic gap via a bidirectional set of synchronous and asynchronous communication channels. Unlike existing virtualization interfaces, symbiotic virtualization places an equal emphasis on both semantic richness and legacy compatibility. The goal of symbiotic virtualization is to introduce a virtualization interfaces that provide access to high level semantic information while still retaining the universal compatibility of a virtual hardware interface. Symbiotic virtualization is an approach to designing VMMs and OSes such that both support, but neither requires, the other. A symbiotic OS targets a native hardware interface, but also exposes a software interface, usable by a symbiotic VMM, if present, to optimize performance and increase functionality. Symbiotic virtualization is neither full system virtualization nor paravirtualization, however it can be used with either approach. Symbiotic virtualization introduces OS changes that facilitate rich information gathering by the VMM, and focuses on the VMM's functional interface to the OS and not the inverse. A symbiotically virtualized architecture supports multiple symbiotic interfaces. Symbiotic interfaces are optional for the guest, and a guest which does use it can also run on non-symbiotic VMMs or raw hardware without any changes. A symbiotic OS exposes two types of interfaces. Passive interfaces allow a symbiotic VMM to directly access internal guest state. This information is exposed directly to the VMM, via an asynchronous communication channel. This interface has extremely low overhead, however its asynchronous nature limits the kind of information that can be accessed in this way. Functional interfaces allow a symbiotic VMM to invoke the guest directly in order to request that the guest perform an operation on behalf of the VMM. These interfaces impose a higher overhead than passive interfaces, but allow for synchronous invocation and support more complex state queries. This dissertation will examine symbiotic virtualization and two symbiotic interfaces: SymSpy and SymCall. I will also describe SymMod an interface that allows a VMM to dynamically create additional symbiotic interfaces at runtime. These interfaces allow for both passive, asynchronous and active, synchronous communication between guest and VMM. I have implemented a symbiotic virtualization framework inside Palacios, an OS independent embeddable virtual machine monitor that I have led the development of. Palacios is a wholly new VMM architecture designed specifically to target areas that have been ill served by existing virtualization tools, namely high performance computing, architecture and education. Palacios supports multiple physical host and virtual guest environments, is compatible with both AMD SVM and Intel VT architectures, and has been evaluated on commodity Ethernet based servers, a high end Infiniband cluster, as well as Red Storm development cages consisting of Cray XT nodes. Palacios also supports the virtualization of a diverse set of guest OS environments, including commodity Linux and other OS distributions, modern Linux kernels, and several lightweight HPC OSes such as CNL, Catamount, and the Kitten Lightweight Kernel.